Large Language Model Performance on Practice Epilepsy Board Examinations.

This diagnostic study examines whether large language models are able to pass practice licensing examinations for epilepsy.

Inductive reasoning with large language models: a simulated randomized controlled trial for epilepsy.

Importance: The analysis of electronic medical records at scale to learn from clinical experience is currently very challenging. The integration of artificial intelligence (AI), specifically foundational large language models (LLMs), into an analysis pipeline may overcome some of the current limitations of modest input sizes, inaccuracies, biases, and incomplete knowledge bases. Objective: To explore the effectiveness of using an LLM for generating realistic clinical data and other LLMs for summarizing and synthesizing information in a model system, simulating a randomized clinical trial (RCT) in epilepsy to demonstrate the potential of inductive reasoning via medical chart review. Design: An LLM-generated simulated RCT based on a RCT for treatment with an antiseizure medication, cenobamate, including a placebo arm and a full-strength drug arm, evaluated by an LLM-based pipeline versus a human reader. Setting: Simulation based on realistic seizure diaries, treatment effects, reported symptoms and clinical notes generated by LLMs with multiple different neurologist writing styles. Participants: Simulated cohort of 240 patients, divided 1:1 into placebo and drug arms. Intervention: Utilization of LLMs for the generation of clinical notes and for the synthesis of data from these notes, aiming to evaluate the efficacy and safety of cenobamate in seizure control either with a human evaluator or AI-pipeline. Measures: The AI and human analysis focused on identifying the number of seizures, symptom reports, and treatment efficacy, with statistical analysis comparing the 50%-responder rate and median percentage change between the placebo and drug arms, as well as side effect rates in each arm. Results: AI closely mirrored human analysis, demonstrating the drug's efficacy with marginal differences (<3%) in identifying both drug efficacy and reported symptoms. Conclusions and Relevance: This study showcases the potential of LLMs accurately simulate and analyze clinical trials. Significantly, it highlights the ability of LLMs to reconstruct essential trial elements, identify treatment effects, and recognize reported symptoms, within a realistic clinical framework. The findings underscore the relevance of LLMs in future clinical research, offering a scalable, efficient alternative to traditional data mining methods without the need for specialized medical language training.

Exercise, medication adherence, and the menstrual cycle: How much do these change seizure risk?

People with epilepsy can experience tremendous stress from the uncertainty of when a seizure will occur. Three factors deemed important because of their potential influence on seizure risk are exercise, medication adherence, and the menstrual cycle. A narrative review was conducted through PubMed searching for relevant articles on how seizure risk is modified by 1) exercise, 2) medication adherence, and 3) the menstrual cycle. There was no consensus about the impact of exercise on seizure risk. Studies about medication nonadherence suggested an increase in seizure risk, but there was not a sufficient amount of data for a definitive conclusion. Most studies about the menstrual cycle reported an increase in seizures connected to a specific aspect of the menstrual cycle. No definitive studies were available to quantify this impact precisely. All three triggers reviewed had gaps in the research available, making it not yet possible to definitively quantify a relationship to seizure risk. More quantitative prospective studies are needed to ascertain the extent to which these triggers modify seizure risk.

Focal Cooling for Drug-Resistant Epilepsy: A Review.

Importance: Epilepsy affects at least 1.2% of the population, with one-third of cases considered to be drug-resistant epilepsy (DRE). For these cases, focal cooling therapy may be a potential avenue for treatment, offering hope to people with DRE for freedom from seizure. The therapy leverages neuroscience and engineering principles to deliver a reversible treatment unhindered by pharmacology. Observations: Analogous to (but safer than) the use of global cooling in postcardiac arrest and neonatal ischemic injury, extensive research supports the premise that focal cooling as a long-term treatment for epilepsy could be effective. The potential advantages of focal cooling are trifold: stopping epileptiform discharges, seizures, and status epilepticus safely across species (including humans). Conclusions and Relevance: This Review presents the most current evidence supporting focal cooling in epilepsy. Cooling has been demonstrated as a potentially safe and effective treatment modality for DRE, although it is not yet ready for use in humans outside of randomized clinical trials. The Review will also offer a brief overview of the technical challenges related to focal cooling in humans, including the optimal device design and cooling parameters.

Recognizing and refuting the myth of tongue swallowing during a seizure.

OBJECTIVE: There is a harmful myth that persists in modern culture that one should place objects into a seizing person's mouth to prevent "swallowing the tongue." Despite expert guidelines against this, the idea remains alive in popular media and public belief. We aimed to investigate the myth's origins and discredit it. METHODS: A medical and popular literature review was conducted for the allusions to "swallowing one's tongue" and practice recommendations for and against placing objects into a seizing person's mouth. Current prevalence of these beliefs and relevant anatomy and physiology were summarised. RESULTS: The first English language allusions to placing objects in a patient's mouth occurred in the mid-19th century, and the first allusions to swallowing one's tongue during a seizure occurred in the late 19th century. By the mid-20th century, it was clear that some were recommending against the practice of placing objects in a patient's mouth to prevent harm. Relatively recent popular literature and film continue to portray incorrect seizure first aid through at least 2013. There is ample modern literature confirming the anatomical impossibility of swallowing one's tongue and confirming the potential harm of putting objects in a patient's mouth. CONCLUSION: One cannot swallow their tongue during a seizure. Foreign objects should not be placed into a seizing person's mouth. We must continue to disseminate these ideas to our patients and colleagues. As neurologists, we have an obligation to champion safe practices for our patients, especially when popular media and culture continue to propagate dangerous ones.

Development and Validation of Forecasting Next Reported Seizure Using e-Diaries.

OBJECTIVE: There are no validated methods for predicting the timing of seizures. Using machine learning, we sought to forecast 24-hour risk of self-reported seizure from e-diaries. METHODS: Data from 5,419 patients on SeizureTracker.com (including seizure count, type, and duration) were split into training (3,806 patients/1,665,215 patient-days) and testing (1,613 patients/549,588 patient-days) sets with no overlapping patients. An artificial intelligence (AI) program, consisting of recurrent networks followed by a multilayer perceptron ("deep learning" model), was trained to produce risk forecasts. Forecasts were made from a sliding window of 3-month diary history for each day of each patient's diary. After training, the model parameters were held constant and the testing set was scored. A rate-matched random (RMR) forecast was compared to the AI. Comparisons were made using the area under the receiver operating characteristic curve (AUC), a measure of binary discrimination performance, and the Brier score, a measure of forecast calibration. The Brier skill score (BSS) measured the improvement of the AI Brier score compared to the benchmark RMR Brier score. Confidence intervals (CIs) on performance statistics were obtained via bootstrapping. RESULTS: The AUC was 0.86 (95% CI = 0.85-0.88) for AI and 0.83 (95% CI = 0.81-0.85) for RMR, favoring AI (p < 0.001). Overall (all patients combined), BSS was 0.27 (95% CI = 0.23-0.31), also favoring AI (p < 0.001). INTERPRETATION: The AI produced a valid forecast superior to a chance forecaster, and provided meaningful forecasts in the majority of patients. Future studies will be needed to quantify the clinical value of these forecasts for patients. ANN NEUROL 2020;88:588-595.

Placebo in epilepsy.

Placebos impact epilepsy in a number of ways. Through randomized clinical trials, explicit clinical use, and also through implicit clinical use, placebos play a role in epilepsy. This chapter will discuss the reasons placebo is used, the determinants of placebo response in epilepsy, observations about placebo specific to epilepsy, and ways in which clinical trial design is impacted by placebo.

Natural variability in seizure frequency: Implications for trials and placebo.

BACKGROUND: Changes in patient-reported seizure frequencies are the gold standard used to test efficacy of new treatments in randomized controlled trials (RCTs). Recent analyses of patient seizure diary data suggest that the placebo response may be attributable to natural fluctuations in seizure frequency, though the evidence is incomplete. Here we develop a data-driven statistical model and assess the impact of the model on interpretation of placebo response. METHODS: A synthetic seizure diary generator matching statistical properties seen across multiple epilepsy diary datasets was constructed. The model was used to simulate the placebo arm of 5000 RCTs. A meta-analysis of 23 historical RCTs was compared to the simulations. RESULTS: The placebo 50 %-responder rate (RR50) was 27.3 ± 3.6 % (simulated) and 21.1 ± 10.0 % (historical). The placebo median percent change (MPC) was 22.0 ± 6.0 % (simulated) and 16.7 ± 10.3 % (historical). CONCLUSIONS: A statistical model of daily seizure count generation which incorporates quantities related to the natural fluctuations of seizure count data produces a placebo response comparable to those seen in historical RCTs. This model may be useful in better understanding the seizure count fluctuations seen in patients in other clinical settings.

Using mobile location data in biomedical research while preserving privacy.

Location data are becoming easier to obtain and are now bundled with other metadata in a variety of biomedical research applications. At the same time, the level of sophistication required to protect patient privacy is also increasing. In this article, we provide guidance for institutional review boards (IRBs) to make informed decisions about privacy protections in protocols involving location data. We provide an overview of some of the major categories of technical algorithms and medical-legal tools at the disposal of investigators, as well as the shortcomings of each. Although there is no "one size fits all" approach to privacy protection, this article attempts to describe a set of practical considerations that can be used by investigators, journal editors, and IRBs.

Is seizure frequency variance a predictable quantity?

Background: There is currently no formal method for predicting the range expected in an individual's seizure counts. Having access to such a prediction would be of benefit for developing more efficient clinical trials, but also for improving clinical care in the outpatient setting. Methods: Using three independently collected patient diary datasets, we explored the predictability of seizure frequency. Three independent seizure diary databases were explored: SeizureTracker (n = 3016), Human Epilepsy Project (n = 93), and NeuroVista (n = 15). First, the relationship between mean and standard deviation in seizure frequency was assessed. Using that relationship, a prediction for the range of possible seizure frequencies was compared with a traditional prediction scheme commonly used in clinical trials. A validation dataset was obtained from a separate data export of SeizureTracker to further verify the predictions. Results: A consistent mathematical relationship was observed across datasets. The logarithm of the average seizure count was linearly related to the logarithm of the standard deviation with a high correlation (R2 > 0.83). The three datasets showed high predictive accuracy for this log-log relationship of 94%, compared with a predictive accuracy of 77% for a traditional prediction scheme. The independent validation set showed that the log-log predicted 94% of the correct ranges while the RR50 predicted 77%. Conclusion: Reliably predicting seizure frequency variability is straightforward based on knowledge of mean seizure frequency, across several datasets. With further study, this may help to increase the power of RCTs, and guide clinical practice.

Does accounting for seizure frequency variability increase clinical trial power?

OBJECTIVE: Seizure frequency variability is associated with placebo responses in randomized controlled trials (RCT). Increased variability can result in drug misclassification and, hence, decreased statistical power. We investigated a new method that directly incorporated variability into RCT analysis, ZV. METHODS: Two models were assessed: the traditional 50%-responder rate (RR50), and the variability-corrected score, ZV. Each predicted seizure frequency upper and lower limits using prior seizures. Accuracy was defined as percentage of time-intervals when the observed seizure frequencies were within the predicted limits. First, we tested the ZV method on three datasets (SeizureTracker: n=3016, Human Epilepsy Project: n=107, and NeuroVista: n=15). An additional independent SeizureTracker validation dataset was used to generate a set of 200 simulated trials each for 5 different sample sizes (total N=100 to 500 by 100), assuming 20% dropout and 30% drug efficacy. "Power" was determined as the percentage of trials successfully distinguishing placebo from drug (p<0.05). RESULTS: Prediction accuracy across datasets was, ZV: 91-100%, RR50: 42-80%. Simulated RCT ZV analysis achieved >90% power at N=100 per arm while RR50 required N=200 per arm. SIGNIFICANCE: ZV may increase the statistical power of an RCT relative to the traditional RR50.

Response to placebo in clinical epilepsy trials--Old ideas and new insights.

Randomized placebo-controlled trials are a mainstay of modern clinical epilepsy research; the success or failure of innovative therapies depends on proving superiority to a placebo. Consequently, understanding what drives response to placebo (including the "placebo effect") may facilitate evaluation of new therapies. In this review, part one will explore observations about placebos specific to epilepsy, including the relatively higher placebo response in children, apparent increase in placebo response over the past several decades, geographic variation in placebo effect, relationship to baseline epilepsy characteristics, influence of nocebo on clinical trials, the possible increase in (SUDEP) in placebo arms of trials, and patterns that placebo responses appear to follow in individual patients. Part two will discuss the principal causes of placebo responses, including regression to the mean, anticipation, classical conditioning, the Hawthorne effect, expectations from symbols, and the natural history of disease. Included in part two will be a brief overview of recent advances using simulations from large datasets that have afforded new insights into causes of epilepsy-related placebo responses. In part three, new developments in study design will be explored, including sequential parallel comparison, two-way enriched design, time to pre-randomization, delayed start, and cohort reduction techniques.

Serial headache drawings by children with migraine: correlation with clinical headache status.

Children's artistic self-depictions of headache provide valuable insights into their experience of pain and aid in the diagnostic differentiation of headache types. In a previous study, we compared the clinical diagnosis (gold standard) and artistic diagnosis of headaches in 226 children. In approximately 90% of cases, the drawing predicted the clinical diagnosis of migraine versus nonmigraine headache correctly. In the present study, we explored whether headache drawings correlate with clinical improvement after treatment in children with migraine headaches followed longitudinally. Children seen in the Pediatric Neurology Clinic with the chief complaint of headache were asked to draw a picture of what their headache feels like. On subsequent clinic visits, children with the clinical diagnosis of migraine were asked to draw another picture depicting their current headache. The two drawings were compared to assess whether there was improvement; this "artistic response" was then correlated with the child's clinical status (ie, whether the headaches were improved clinically). One hundred eleven children (66 girls, 45 boys) participated in the study, with a mean interval of 5.3 +/- 2.3 (standard error of the mean) months between the first and second visits. The mean age at the first visit was 11.6 +/- 3.1 years. The raters agreed that serial drawings were both improved or both not improved in 99 of the 111 cases (89%; interrater reliability kappa score of 0.767). Fifty-three children had improvements in their headaches and drawings, 3 children had an improved drawing but no clinical headache improvement, 32 children had no improvement in either their drawing or clinical headaches, and 11 children had improved headaches but no improvement in their drawing. The sensitivity of the drawings for clinical improvement was 0.83, and the specificity was 0.91. The predictive value of an improved headache drawing for an improved clinical response was 0.946. There was no correlation between specific treatment modality and artistic response. We concluded that children's headache drawings are a useful adjunct for the diagnosis of headache type and provide valuable insights into their experience of pain. The present data show that headache drawings can be used longitudinally to provide additional information about the clinical course. The technique is simple, inexpensive, and enjoyable for children and can be applied in a variety of clinical settings.